An Online Ensemble of Classifiers

Along with the explosive increase of data and information, incremental learning ability has become more and more important for machine learning approaches. The online algorithms try to forget irrelevant information instead of synthesizing all available information (as opposed to classic batch learning algorithms). Nowadays, combining classifiers is proposed as a new direction for the improvement of the classification accuracy. However, most ensemble algorithms operate in batch mode. For this reason, we propose an online ensemble of classifiers that combines an incremental version of Naive Bayes, the Voted Perceptron and the Winnow algorithms using the voting methodology. We performed a largescale comparison of the proposed ensemble with other state-of-the-art algorithms on several datasets and we took better accuracy in most cases.